SherbyNet - /page/wikibook/entry.mu

<!DOCTYPE html>

<title>Least-squares function approximation</title>

</head>

Least-squares function approximation

</h1>

</header>

In <a href="Mathematics" title="Mathematics">mathematics</a>, least squares function approximation applies the principle of <a href="Least_squares" title="Least squares">least squares</a> to <a href="Function_approximation" title="Function approximation">function approximation</a>, by means of a weighted sum of other functions. The best approximation can be defined as that which minimizes the difference between the original function and the approximation; for a least-squares approach the quality of the approximation is measured in terms of the squared differences between the two.

<div class="mw-heading mw-heading2"><h2 id="Functional_analysis">Functional analysis</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .hatnote{font-style:italic}.mw-parser-output div.hatnote{padding-left:1.6em;margin-bottom:0.5em}.mw-parser-output .hatnote i{font-style:normal}.mw-parser-output .hatnote+link+.hatnote{margin-top:-0.5em}@media print{body.ns-0 .mw-parser-output .hatnote{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><div role="note" class="hatnote navigation-not-searchable">See also: <a href="Fourier_series" title="Fourier series">Fourier series</a> and <a href="Generalized_Fourier_series" title="Generalized Fourier series">Generalized Fourier series</a></div>

A generalization to approximation of a data set is the approximation of a function by a sum of other functions, usually an <a href="Orthogonal_functions" title="Orthogonal functions">orthogonal set</a>:<a href="#cite_note-Lanczos-1">[1]</a>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f(x)\approx f_{n}(x)=a_{1}\phi _{1}(x)+a_{2}\phi _{2}(x)+\cdots +a_{n}\phi _{n}(x),\ }</annotation>

</semantics>

</math><img src="./2730998b2ab2ab75c4290b858ae375d2c0f5a98f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:52.887ex; height:2.843ex;" alt="{\displaystyle f(x)\approx f_{n}(x)=a_{1}\phi _{1}(x)+a_{2}\phi _{2}(x)+\cdots +a_{n}\phi _{n}(x),\ }" loading="lazy"></dd></dl>

with the set of functions {<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \ \phi _{j}(x)}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \ \phi _{j}(x)}</annotation>

</semantics>

</math><img src="./f3628d9493dcc804fc40f41bc94a1711c38bfa97.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:6.015ex; height:3.009ex;" alt="{\displaystyle \ \phi _{j}(x)}" loading="lazy">} an <a href="Orthonormal_set" class="mw-redirect" title="Orthonormal set">orthonormal set</a> over the interval of interest, say [a, b]: see also <a href="Fej%C3%A9r's_theorem" title="Fejér's theorem">Fejér's theorem</a>. The coefficients {<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \ a_{j}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \ a_{j}}</annotation>

</semantics>

</math><img src="./41349b34551d674414c7d0c1aca88efdd926ce53.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:2.72ex; height:2.343ex;" alt="{\displaystyle \ a_{j}}" loading="lazy">} are selected to make the magnitude of the difference ||f − fn||2 as small as possible. For example, the magnitude, or norm, of a function g (x ) over the interval [a, b] can be defined by:<a href="#cite_note-Folland-2">[2]</a>

<msup>

<mrow>

<mrow>

</mrow>

</mrow>

</msubsup>

<msup>

</mrow>

</msup>

</mrow>

</mrow>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \|g\|=\left(\int _{a}^{b}g^{*}(x)g(x)\,dx\right)^{1/2}}</annotation>

</semantics>

</math><img src="./e2ba10510391613f33f39f05a46746ad4f074e05.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.505ex; width:28.947ex; height:7.009ex;" alt="{\displaystyle \|g\|=\left(\int _{a}^{b}g^{*}(x)g(x)\,dx\right)^{1/2}}" loading="lazy"></dd></dl>

where the ‘*’ denotes <a href="Complex_conjugate" title="Complex conjugate">complex conjugate</a> in the case of complex functions. The extension of Pythagoras' theorem in this manner leads to <a href="Function_space" title="Function space">function spaces</a> and the notion of <a href="Lebesgue_measure" title="Lebesgue measure">Lebesgue measure</a>, an idea of “space” more general than the original basis of Euclidean geometry. The { <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \phi _{j}(x)\ }">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \phi _{j}(x)\ }</annotation>

</semantics>

</math><img src="./5a9210925644f147769211a89f5577897dde3f20.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:6.015ex; height:3.009ex;" alt="{\displaystyle \phi _{j}(x)\ }" loading="lazy"> } satisfy <a href="Orthogonal" class="mw-redirect" title="Orthogonal">orthonormality relations</a>:<a href="#cite_note-Folland2-3">[3]</a>

</mrow>

</mrow>

</msubsup>

</mrow>

</mrow>

</msubsup>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \int _{a}^{b}\phi _{i}^{*}(x)\phi _{j}(x)\,dx=\delta _{ij},}</annotation>

</semantics>

</math><img src="./1701b20e55f1d0d0efec947942e024bd7995d58e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.338ex; width:23.989ex; height:6.343ex;" alt="{\displaystyle \int _{a}^{b}\phi _{i}^{*}(x)\phi _{j}(x)\,dx=\delta _{ij},}" loading="lazy"></dd></dl>

where δij is the <a href="Kronecker_delta" title="Kronecker delta">Kronecker delta</a>. Substituting function fn into these equations then leads to

the n-dimensional <a href="Pythagorean_theorem" title="Pythagorean theorem">Pythagorean theorem</a>:<a href="#cite_note-Wood-4">[4]</a>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</msup>

</mrow>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</mrow>

</msup>

</mrow>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</mrow>

</msup>

</mrow>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \|f_{n}\|^{2}=|a_{1}|^{2}+|a_{2}|^{2}+\cdots +|a_{n}|^{2}.\,}</annotation>

</semantics>

</math><img src="./3ebfa6e0f7374a47f4c64c3a6b6dd0c17b34e3d7.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:35.174ex; height:3.343ex;" alt="{\displaystyle \|f_{n}\|^{2}=|a_{1}|^{2}+|a_{2}|^{2}+\cdots +|a_{n}|^{2}.\,}" loading="lazy"></dd></dl>

The coefficients {aj} making ||f − fn||2 as small as possible are found to be:<a href="#cite_note-Lanczos-1">[1]</a>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</msubsup>

</mrow>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle a_{j}=\int _{a}^{b}\phi _{j}^{*}(x)f(x)\,dx.}</annotation>

</semantics>

</math><img src="./3c5a26ff78be391fee847ef498591c9cb2f59445.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.338ex; width:22.603ex; height:6.343ex;" alt="{\displaystyle a_{j}=\int _{a}^{b}\phi _{j}^{*}(x)f(x)\,dx.}" loading="lazy"></dd></dl>

The generalization of the n-dimensional Pythagorean theorem to infinite-dimensional <a href="Real_number" title="Real number">real</a> inner product spaces is known as <a href="Parseval's_identity" title="Parseval's identity">Parseval's identity</a> or Parseval's equation.<a href="#cite_note-Folland3-5">[5]</a> Particular examples of such a representation of a function are the <a href="Fourier_series" title="Fourier series">Fourier series</a> and the <a href="Generalized_Fourier_series" title="Generalized Fourier series">generalized Fourier series</a>.

<div class="mw-heading mw-heading2"><h2 id="Further_discussion">Further discussion</h2></div>

<div class="mw-heading mw-heading3"><h3 id="Using_linear_algebra">Using linear algebra</h3></div>

It follows that one can find a "best" approximation of another function by minimizing the area between two functions, a continuous function <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f}</annotation>

</semantics>

</math><img src="./132e57acb643253e7810ee9702d9581f159a1c61.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.279ex; height:2.509ex;" alt="{\displaystyle f}" loading="lazy"> on <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle [a,b]}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle [a,b]}</annotation>

</semantics>

</math><img src="./9c4b788fc5c637e26ee98b45f89a5c08c85f7935.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:4.555ex; height:2.843ex;" alt="{\displaystyle [a,b]}" loading="lazy"> and a function <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle g\in W}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle g\in W}</annotation>

</semantics>

</math><img src="./64ffd704ef419b6b79cb07de8e5f579da4baa530.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:6.392ex; height:2.509ex;" alt="{\displaystyle g\in W}" loading="lazy"> where <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle W}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle W}</annotation>

</semantics>

</math><img src="./54a9c4c547f4d6111f81946cad242b18298d70b7.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.435ex; height:2.176ex;" alt="{\displaystyle W}" loading="lazy"> is a subspace of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle C[a,b]}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle C[a,b]}</annotation>

</semantics>

</math><img src="./7c1625217aad8c105c50c975599e45192b2bfbec.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:6.321ex; height:2.843ex;" alt="{\displaystyle C[a,b]}" loading="lazy">:

</mrow>

</mrow>

</mrow>

</msubsup>

<mrow>

<mrow>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\text{Area}}=\int _{a}^{b}\left\vert f(x)-g(x)\right\vert \,dx,}</annotation>

</semantics>

</math><img src="./23ad7c775152df9adcb00762213c7ceefee03e75.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.338ex; width:28.51ex; height:6.343ex;" alt="{\displaystyle {\text{Area}}=\int _{a}^{b}\left\vert f(x)-g(x)\right\vert \,dx,}" loading="lazy"></dd></dl>

all within the subspace <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle W}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle W}</annotation>

</semantics>

</math><img src="./54a9c4c547f4d6111f81946cad242b18298d70b7.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.435ex; height:2.176ex;" alt="{\displaystyle W}" loading="lazy">. Due to the frequent difficulty of evaluating integrands involving absolute value, one can instead define

</mrow>

</mrow>

</msubsup>

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \int _{a}^{b}[f(x)-g(x)]^{2}\,dx}</annotation>

</semantics>

</math><img src="./0c1b945f1142ff48dc9110b80d95539993579c74.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.338ex; width:20.195ex; height:6.343ex;" alt="{\displaystyle \int _{a}^{b}[f(x)-g(x)]^{2}\,dx}" loading="lazy"></dd></dl>

as an adequate criterion for obtaining the least squares approximation, function <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle g}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle g}</annotation>

</semantics>

</math><img src="./d3556280e66fe2c0d0140df20935a6f057381d77.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.116ex; height:2.009ex;" alt="{\displaystyle g}" loading="lazy">, of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f}</annotation>

</semantics>

</math><img src="./132e57acb643253e7810ee9702d9581f159a1c61.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.279ex; height:2.509ex;" alt="{\displaystyle f}" loading="lazy"> with respect to the inner product space <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle W}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle W}</annotation>

</semantics>

</math><img src="./54a9c4c547f4d6111f81946cad242b18298d70b7.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.435ex; height:2.176ex;" alt="{\displaystyle W}" loading="lazy">.

As such, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \lVert f-g\rVert ^{2}}">

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \lVert f-g\rVert ^{2}}</annotation>

</semantics>

</math><img src="./fbba44eaed83850f648f6dc246a5035542b8c1c1.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:8.614ex; height:3.176ex;" alt="{\displaystyle \lVert f-g\rVert ^{2}}" loading="lazy"> or, equivalently, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \lVert f-g\rVert }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \lVert f-g\rVert }</annotation>

</semantics>

</math><img src="./d7524e3b8fe4918538a81912386d9b1a6c4d42eb.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:7.56ex; height:2.843ex;" alt="{\displaystyle \lVert f-g\rVert }" loading="lazy">, can thus be written in vector form:

</mrow>

</mrow>

</msubsup>

<msup>

</mrow>

</msup>

<mrow>

<mrow>

</mrow>

</mrow>

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \int _{a}^{b}[f(x)-g(x)]^{2}\,dx=\left\langle f-g,f-g\right\rangle =\lVert f-g\rVert ^{2}.}</annotation>

</semantics>

</math><img src="./29cdda7506886c0e092566a8dd56b2c6004df2bd.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.338ex; width:48.966ex; height:6.343ex;" alt="{\displaystyle \int _{a}^{b}[f(x)-g(x)]^{2}\,dx=\left\langle f-g,f-g\right\rangle =\lVert f-g\rVert ^{2}.}" loading="lazy"></dd></dl>

In other words, the least squares approximation of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f}</annotation>

</semantics>

</math><img src="./132e57acb643253e7810ee9702d9581f159a1c61.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.279ex; height:2.509ex;" alt="{\displaystyle f}" loading="lazy"> is the function <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle g\in {\text{ subspace }}W}">

<mtext> subspace </mtext>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle g\in {\text{ subspace }}W}</annotation>

</semantics>

</math><img src="./4423b5499eeae726eebec0eedff66895ceb6fde2.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:16.491ex; height:2.509ex;" alt="{\displaystyle g\in {\text{ subspace }}W}" loading="lazy"> closest to <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f}</annotation>

</semantics>

</math><img src="./132e57acb643253e7810ee9702d9581f159a1c61.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.279ex; height:2.509ex;" alt="{\displaystyle f}" loading="lazy"> in terms of the inner product <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \left\langle f,g\right\rangle }">

<mrow>

<mrow>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \left\langle f,g\right\rangle }</annotation>

</semantics>

</math><img src="./94c7e3388b044fc8e84567bfb1c58ded7170067c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:5.238ex; height:2.843ex;" alt="{\displaystyle \left\langle f,g\right\rangle }" loading="lazy">. Furthermore, this can be applied with a theorem:

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f}</annotation>

</semantics>

</math><img src="./132e57acb643253e7810ee9702d9581f159a1c61.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.279ex; height:2.509ex;" alt="{\displaystyle f}" loading="lazy"> be continuous on <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle [a,b]}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle [a,b]}</annotation>

</semantics>

</math><img src="./9c4b788fc5c637e26ee98b45f89a5c08c85f7935.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:4.555ex; height:2.843ex;" alt="{\displaystyle [a,b]}" loading="lazy">, and let <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle W}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle W}</annotation>

</semantics>

</math><img src="./54a9c4c547f4d6111f81946cad242b18298d70b7.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.435ex; height:2.176ex;" alt="{\displaystyle W}" loading="lazy"> be a finite-dimensional subspace of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle C[a,b]}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle C[a,b]}</annotation>

</semantics>

</math><img src="./7c1625217aad8c105c50c975599e45192b2bfbec.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:6.321ex; height:2.843ex;" alt="{\displaystyle C[a,b]}" loading="lazy">. The least squares approximating function of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f}</annotation>

</semantics>

</math><img src="./132e57acb643253e7810ee9702d9581f159a1c61.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.279ex; height:2.509ex;" alt="{\displaystyle f}" loading="lazy"> with respect to <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle W}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle W}</annotation>

</semantics>

</math><img src="./54a9c4c547f4d6111f81946cad242b18298d70b7.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.435ex; height:2.176ex;" alt="{\displaystyle W}" loading="lazy"> is given by

<mrow>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mrow>

</mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<mrow>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mrow>

</mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<mrow>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mrow>

</mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle g=\left\langle f,{\vec {w}}_{1}\right\rangle {\vec {w}}_{1}+\left\langle f,{\vec {w}}_{2}\right\rangle {\vec {w}}_{2}+\cdots +\left\langle f,{\vec {w}}_{n}\right\rangle {\vec {w}}_{n},}</annotation>

</semantics>

</math><img src="./4f90ed9ad989542f895ccf84d0a9a86417e4b808.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:46.271ex; height:2.843ex;" alt="{\displaystyle g=\left\langle f,{\vec {w}}_{1}\right\rangle {\vec {w}}_{1}+\left\langle f,{\vec {w}}_{2}\right\rangle {\vec {w}}_{2}+\cdots +\left\langle f,{\vec {w}}_{n}\right\rangle {\vec {w}}_{n},}" loading="lazy"></dd></dl></dd>

<dd>where <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle B=\{{\vec {w}}_{1},{\vec {w}}_{2},\dots ,{\vec {w}}_{n}\}}">

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle B=\{{\vec {w}}_{1},{\vec {w}}_{2},\dots ,{\vec {w}}_{n}\}}</annotation>

</semantics>

</math><img src="./8b1c9b2d97b2f361ec6858fd18e51779e827d18a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:21.719ex; height:2.843ex;" alt="{\displaystyle B=\{{\vec {w}}_{1},{\vec {w}}_{2},\dots ,{\vec {w}}_{n}\}}" loading="lazy"> is an orthonormal basis for <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle W}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle W}</annotation>

</semantics>

</math><img src="./54a9c4c547f4d6111f81946cad242b18298d70b7.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.435ex; height:2.176ex;" alt="{\displaystyle W}" loading="lazy">.</dd></dl>

<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}

/* end https://en.wikipedia.org/ */

</style><div class="reflist">

/* start https://en.wikipedia.org/ */

.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}

/* end https://en.wikipedia.org/ */

</style><cite id="CITEREFCornelius_Lanczos1988" class="citation book cs1">Cornelius Lanczos (1988). <a rel="nofollow" class="external text" href="https://books.google.com/books?id=6E85hExIqHYC&pg=PA212">Applied analysis</a> (Reprint of 1956 Prentice–Hall ed.). Dover Publications. pp. 212–213. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>0-486-65656-X</bdi>.</cite>

</li>

<cite id="CITEREFGerald_B_Folland2009" class="citation book cs1">Gerald B Folland (2009). <a rel="nofollow" class="external text" href="https://books.google.com/books?id=ix2iCQ-o9x4C&pg=PA69">"Equation 3.14"</a>. Fourier analysis and its application (Reprint of Wadsworth and Brooks/Cole 1992 ed.). American Mathematical Society Bookstore. p. 69. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-0-8218-4790-9</bdi>.</cite>

</li>

<cite id="CITEREFFolland2009" class="citation book cs1">Folland, Gerald B (2009). <a rel="nofollow" class="external text" href="https://books.google.com/books?id=ix2iCQ-o9x4C&pg=PA69">Fourier Analysis and Its Applications</a>. American Mathematical Society. p. 69. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-0-8218-4790-9</bdi>.</cite>

</li>

<cite id="CITEREFDavid_J._Saville,_Graham_R._Wood1991" class="citation book cs1">David J. Saville, Graham R. Wood (1991). <a rel="nofollow" class="external text" href="https://books.google.com/books?id=8ummgMVRev0C&pg=PA30">"§2.5 Sum of squares"</a>. Statistical methods: the geometric approach (3rd ed.). Springer. p. 30. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>0-387-97517-9</bdi>.</cite>

</li>

<cite id="CITEREFGerald_B_Folland2009" class="citation book cs1">Gerald B Folland (2009-01-13). <a rel="nofollow" class="external text" href="https://books.google.com/books?id=ix2iCQ-o9x4C&pg=PA77">"Equation 3.22"</a>. cited work. American Mathematical Soc. p. 77. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-0-8218-4790-9</bdi>.</cite>

</li>

</ol></div></div></div><div><div class="zim-footer">

This article is issued from <a class="external text" title="Last edited on 2023-12-13" href="https://en.wikipedia.org/wiki/?title=Least-squares_function_approximation&oldid=1189632447">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.

</div>

</div></div>

</div>

</main>

</div>

</body></html>